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I.  ABSTRACT  (Miilmvm  JOO  wordi) 

The  Imagery  Exploitation  System/Balanced  Technology  Initiative  (lES/BTI)  is  an 
automated  first  phase  near-real-time  image  exploitation  system  to  suppoVt 
Army  Corps  intelligence  and  electronic  warfare  situation  development •  target 
development  and  target  acquisition.  TES/BTI  exploits  synthetic  aperture  radar 
(SAR)  and  Infared  (IR)  imagery  and  annotates  the  presence  and  type  of  military 
units  (company  size  and  above). 

lES/BTI  employs  a  hierarchlal  reasoning  paradigm,  using  Bayesian  inference  for 
hypotheses  management  and  belief  propagation  to  solve  the  complex  force/terrain/ 
military  situation  and  formation.  Evidence  gathering  actions  evaluate  the  closeness 
of  data  supporting  the  hypotheses  to  the  force  models  using  statistical  metrics 
and/or  expert  system  rules. 

lES/BTI  has  processed  data  from  two  military  theaters  and  lES/BTI  has  learned  lesson? 
concerning  which  methods  have  worked  well  and  which  ideas  haven't  met  expectations. 
These  lessons  learned  are  presented. 
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U.S  Army  Tqx)gr^hic  Engineering  Center 
Fort  Belvoir,  Virginia  22060-5546 


ABSTRACT 

The  Imagery  Exploitation  System  /  Balanced  Technology  Initiative  (lES/BTI)  is  a  first 
phase  near-real  time  image  exploitation  system  to  support  Army  Corps  intelligence  and 
electronic  warfare  (lEW)  situation  development,  target  development  and  target  acquisition. 
lES/BTI  exploits  synthetic  aperture  radar  (SAR)  and  infmed  (IR)  imagay  and  annotates 
the  presence  and  t]^  (artillery,  armcx-,  etc)  of  military  units  (company  size  and  above). 

lES/BTI  employs  a  coarse-to-fine  hierarchical  reasoning  paradigm,  using  Bayesian 
inference  for  hypotheses  management  and  belief  propagation  to  solve  the  complex 
forceAerrain/nulitary  situation  image  understanding  problem.  Military  fences  are  modeled 
at  multiple  levels  of  abstraction  representing  fence  hierarchy,  situatiem  and  formatiem. 
Evidence  gathering  actions  evaluate  the  closeness  of  data  supporting  the  hypotheses  to  the 
force  models  using  statistical  metrics  and/or  expert  system  rules.  Some  rules  are  complex 
dynamically  adapting  a  static  fence  model  to  the  local  terrain  and  threat  situation.  A  uniform 
certainty  cdculus  defines  how  evidence  frenn  the  model  match  is  applied  to  the  hypotheses. 

In  lES/BTI,  hypotheses  are  stored  in  the  Bayes  network  a  multidimensional 
hierarchical  tree  structure  where  hypotheses  and  associated  evidence  are  probabilistically 
linked  across  dimensions.  The  state  of  force  hypotheses  stored  in  the  Bayes  network  are 
monitored  by  decision  theoretic  control  (DTC)  which  opportunistically  spawns  actions  to 
evaluate,  refine,  spawn  and/or  prune  force  hypotheses.  lES/BTI  completes  processing 
when  DTC  determines  the  sufficient  evidence  has  been  gathered  to  answer  the  exploitation 
request  and  image  aruiotations  are  produced. 

In  a  blind  test  on  33  images;  2  regiments,  45  battalions  and  36  companies  of  acmal 
enemy  forces  were  identified  by  lES/BTI. 
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IMAGERY  EXPLOITATION  SYSTEM  /  BALANCED  TECHNOLOGY  INITIATIVE 


Frederick  H.  Esch 


U.S.  Army  Topographic  Engineering  Center 
Fort  Belvoir,  Virginia  22060-5546 


INTRODUCTION 

The  Imagery  Exploitation  System  /  Balanced  Technology  Initiative  (lES/BTI)  is  a  first 
phase  near-red  time  image  exploitation  system  to  support  Army  Corps  Intelligence 
Electronic  Warfare  (IE>^  situation  development,  target  development  and  target  acquisition. 
lES/BTI  exploits  synthetic  aperture  radar  (SAR)  and  infirared  (IR)  imagery  and  annotates 
the  presence  and  type  (artillery,  armor,  etc)  of  t^itary  units  (company  size  and  above). 

The  lES/BTI  Program's  historical  foundation  includes  the  Advanced  Digital  Radar 
Imagery  exploitation  System  (ADRIES)  and  Model-based  Image  Sensor  Target 
Exploitation  and  Recognition  (MISTER)  programs  which  provided  a  technical  foundation 
for  automated  exploitation  of  tactical  SAR. 

lES/BTI  is  sponsored  by  the  Balanced  Technology  Initiative  with  additional  support 
from  the  U.S.  Army  Topographic  Engineering  Onter  (TEC),  Army  Space  Programs 
Office  (ASPO),  Defence  Advanced  Research  Projects  Agency  (DARPA),  Undersecretary 
of  Army  for  Research  and  Development  (SARDA)  aixi  Wright  Avionics  Laboratory. 
EES/BTl  is  being  developed  by  the  U.S.  Army  Topographic  Engineering  Center  (formerly 
ETL). 


MISSION 

lES/BTI  mission  is  to  provide  an  automated  image  exploitation  system  for  situadcxi 
analysis  with  a  multi-sensor  (ASARS,  other  SAR,  IR)  capability,  provide  near-real  time 
processing  in  an  operational  (tactical)  setting,  ccmplete  the  exploitation  task  for  a  ten 
nautical  mile  (nm)  by  ten  nm  (10  nm  X  10  nm)  area  in  less  than  five  minutes  (<  5  minutes) 
with  ninety  percent  (90%)  cwrectness  and  act  as  a  fwce  multiplier  fw  first  phase  situation 
analysis  image  analysts  (LA).  Given  that  lES/BTI  will  increase  the  speed  and  accuracy  at 
which  an  LA  can  interpret  an  image,  lES/BTI  will  increase  the  volume  and  image  analyst 
can  interpret  and  therefore  provide  the  Corps  commander  more  knowledge  of  fie 
battlefield. 


Objective 


lES/BTI  objective  is  to  provide  conunanders  more  knowledge  of  the  battlefield  by 
increasing  the  pi^uctivity  of  softcopy  image  analysts.  lES/BTI  will  increase  the  lA's 
productivity  in  two  dimensions  1)  sp^  and  2)  accuracy.  lES/BTI  increases  an  lA's  speed 
by  relieving  the  analyst  of  low>level  time  consunung  ta^  and  by  focusing  the  analyst  on 
probable  forces.  lES/BTI  increases  an  lA's  accuracy  by  bringing  additicmal  information  to 
the  LA's  attention  and  reducing  the  analysts  fatigue  factor.  is  designed  to  solve  the 

broad  area  search  problem. 

Methodology 

lES/BTI  is  designed  to  automatically  perfom  exploitation  before  the  lA  receives  the 
imagery.  The  analyst  is  presented  the  image  and  results  simultaneously.  This  allows  the 
analyst  to  focus  on  the  faces  present  yet  allows  him  to  inspect  all  the  imagery  and  evidence 
present  In  this  data  flow,  lES/BH  technology  resides  between  the  image  formation 
process  and  the  analyst  work  station.  See  Figure  1. 


In  this  methodology,  lES/BTI  performs  the  low-level  vehicle  detection  which  currently 
takes  the  majoity  of  analysts  time.  LES/BTI  also  performs  the  higher  level  reasoning  of 
force  aggregation,  force  typing,  false  hypotheses  rejection  and  merging  of  various  evidence 
sources  and  presenting  the  analyst  the  results.  The  analyst  can  quickly  and  efficiently  focus 
on  the  potential  forces  and  evaluate  them  for  tK^curacy.  Via  this  methodology,  the  analyst 
can  eliminate  major  areas  of  search,  concentrate  oi  the  most  productive  imagery  and  have 
access  to  the  und^lying  evidence. 
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RcQuirements 

Requirements  are  broken  into  two  sections  external  and  internal.  External 
requirements  are  driven  by  published  requirements  and  commanders  needs.  Internal 
requirements  are  driven  by  external  requirements,  program  goals,  good  software  practices 
and  desires  to  conform  to  Army  standards  and  initiatives. 


External  Requirements 

lES/BTI  external  requirements  are  derived  from  the  Army  Intelligence  Master  Plan 
(9/89).  Corps  BEW  timing  and  location  accuracy  requirements  are  given  in  Table  1  and 
Table  2.  The  area  within  the  box  indicates  the  region  where  imagery  is  usually  used. 
Currently,  tactical  electronic  intelligence  (TacElint)  can  meet  the  (^sips  time  requirements 
but  does  not  support  the  Ccnps  location  accuracy,  whereas  imagery  supports  the  accuracy 
requirement. 

Table  1.  lEW  Time  Requirements  at  (Zorps 
Distance  (in  km)  Beyond  PLOT 

Mission  0  -  30  30  -  70  70-150  150  -  300  300-t- 


Mover 

1.8  min 

20  min 

Non-mover 

15  min 

15-60  min 

2hr 

3  hr 

6hr 

TGTDEV 

Mover 

1-2  min 

6.5  min 

6.5  min 

7.5  min 

10  min 

Non-mover 

7.5  min 

7.5  min 

7.5  min 

1.5  hr 

3  hr 

2  min 

2  min 

2min 

N/A 

N/A 

Non-mover 

N/A 

N/A 

2-3  min* 

N/A 

N/A 

*  Prior  to  attack 


Table  2.  lEW  Ijocation  Accuracy  Requirements  at  Q»ps 


Mission 

Distance  (in  km)  Beyond  PLOT 

0  -  30  30  -  70  70-  150 

150  -  300 

300 -t- 

5  km 

5  km 

TGTDEV 

100  m 

175  m 

350  m 

450  m 

450  m 

TGTAQC 

80  m 

100  m 

150  m 

N/A 

N/A 

Internal  Requirements 

lES/BTI  internal  requirements  include  timing,  accuracy,  multi-sensor  ccmsiderations, 
software  standards,  con^liance  with  commercial  of  the  shelf  policy  (CX)TS),  and  theater 
of  operation  portability  requirements.  Top  level  internal  requirements  are  given  in  Table  3. 

Table  3.  Top  Level  lES/BTI  Requirements 

Requirement _ 

Complete  processing  of  image  in  less  than  five  minutes 

Process  in  a  (q)erational  (tactical)  setting 

Provide  multi-sensor  capability  (ASARS2,  other  SAR,  IR) 

Provide  80  m  vehicle  location  accuracy 

Provide  graceful  migration  between  titters  of  operation 

Completion  of  the  image  within  five  minutes  allows  the  analyst  time  (a  minimum  of  8 
minutes  for  situation  analysis)  to  evaluate  the  image  and  lES/BTI  analysis  before  generating 
his  report  This  time  standard  applies  to  images  of  one  hundred  square  nautical  n^es  or 
less  (image  <=  1(X)  sq  nm). 

Before  program  completion,  lES/BTI  will  process  in  a  20-foot  shelter  collocated  with 
the  Imagery  Pressing  Dissemination  System  (IPDS)  in  Europe.  lES/BTI  may  be 
configu^  into  two  standard  19-inch  racks  for  space  constraii^  locations. 

lES/BTI  will  provide  an  ability  to  process  ASARS2,  other  SAR  and  IR.  Outputs  from 
JStars  and  UAV  are  also  under  consideration.  lES/BTTs  intention  is  to  provide  the 
infrastructure  and  paradigm  to  allow  sensors  to  be  gracefully  added  as  automated  vehicle 
detection  when  the  sensor  becomes  available. 

lES/BTI  goal  is  to  provide  target  acquisition  level  location  accuracy.  This  accuracy  is 
obtainable  when  DMA  standard  piquets  are  available  fw  the  area  of  interest  (AOI)  ot 
when  the  ephemeris  registration  provides  this  accuracy. 


The  requirement  to  gracefully  migrate  between  theaters  of  operation  influences  the 
design  and  modularity  of  the  system.  It  is  lES/BTI  intention  to  keep  all  adversary  and 
theater  information  as  parameters  and  data  within  look-up  tables  and/or  databases. 


TECHNICAL  PHILOSOPHY 

lES/BTI  program  technical  philosc^hy  is  to  combine  three  disciplines  1)  image  and 
signal  processing,  2)  knowledge  based  systems  and  3)  terrain  and  military  context  analysis 
into  one  system  that  emulates  the  image  analysis  process.  This  process  uses  the  coaise-to- 
fine  reasoning  of  the  image  analyst  to  overcome  problems  associated  with  the  high  false 
alarm/low  resolution  S  AR  environment  and  allows  complex  reasoning  performed  in  a 
combinatorially  controllable  maimer.  By  using  similar  reasoning  strategies  and  knowledge 
lES/BTI  automatically  emulates  the  image  andyst  process. 

Image  Analyst  Process 

Table  4  outlines  the  steps  in  the  image  analyst  process.  Within  this  process,  the 
analyst  goes  from  vehicle  level  detection  to  higher  level  symbolic  processing  associating 
vehicles  to  units  and  deriving  inter-unit  relationships.  The  analyst  uses  all  clues  available  to 
him  to  analyze  the  image  including  his  prior  knowledge  of  the  area,  knowledge  of  effects  of 
situation  and  terrain  on  deployment  and  knowledge  of  his  adversary’s  tactics.  lES/BTI 
emulation  of  this  process  is  described  in  section  lES/BTI  Processing  Paradigm, 


Table  4.  Steps  in  the  image  analyst  process 
Analyze  tasking  and  plan  exploitation  strategy 
Review 

Q>llateral  cues  (intelligence) 

Previous  coverage 
Terrain/AOI 

Extract  evidence  from  imagery  (detects  targets) 

Group  targets  into  unit  components  (ccwnpany/battalion) 

Use  terrain,  other  intelligence,  military  situation  as  supporting  evidence 
for/against  detections  and  units 

Reason  about  force  structure  and  posture 
Aggregate  units  into  higher  echelon  forces 
Compare  force  layout  against  doctrine 
Predict  locations  of  missing  fOTces^nits 
Integrate  terrain  information 

Generate  a  report _ 


lES/BTI  PROCESSING  PARADIGM 

The  lES/BTI  processing  paradigm  has  three  characteristic  features.  First,  the 
paradigm  is  sensor  and  theater  of  operation  independent  Second,  the  paradigm 
implements  the  lES/BTI  technical  philosophy  and  third,  the  paradigm  emulates  the  lA 
process.  The  lES/BTI  processing  paradigm  is  shown  in  Figure  2. 


IDENTIFIED  FORCES 


Sensor  and  Theater  of  Operation  Independence 


As  shown  in  Hgure  2,  lES/BTl  is  contained  within  the  black  box.  Everything  external 
to  the  black  box  is  independent  of  the  lES/BTI  processing  flow.  This  means  that  changing 
theaters  of  operation  or  adversaries  in  lES/BTI  only  requires  changes  in  databases,  rules 
and  parameters.  Therefore  software  does  not  need  to  be  changed  when  changing  theaters 
of  operation  or  adversaries.  Similarly,  addition  of  new  senors  only  requires  the 
development  of  a  sensw  vehicle  detection  component  and  requires  the  sensors  behavior  be 
understood.  Sensor  dependent  information  is  likewise  stored  in  databases  and  look-up 
tables.  Ccmtrol  flow  which  is  influenced  by  both  theater  of  operation  and  sensor  is 
adjusted  by  picking  the  apprc^mate  DirectX  Acyclic  Gr^h  (DAG)  for  the  run.  The  DAGs 
define  the  fimction^ty  to  be  executed. 

Implementation  of  the  lES/BTI  Processing  PhilosoDhv 

The  implementation  erf  the  lES/BTI  processing  paradigm  has  five  cornerstones  1) 
model  matching,  2)  use  of  multiple  evidence  sources,  3)  coarse-to-fine  reasoning,  4) 
flexible  control  flow  and  5)  Bayesian  inference. 

Models 

Models  are  simply  a  representation  of  knowledge.  In  lES/BTI  models  occur  at  eveiy 
level  of  abstraction  and  all  evaluation  is  done  by  model  matching.  Examples  of  nrusdels  in 
EES/BTI  include  statistical  characterizations  of  vehicle  detections,  artillery  deployment 
patterns,  prohibited  (no-go)  terrain,  force  composition,  force  hierarchy  and  a  radar 
frequency/equipment  correlation.  Several  types  of  models  exist  in  lES/BTI  including 
statistical  analysis,  force  models,  constraint/evaluation  nnodels  and  theoretic  control 
models. 

Statistical  analysis  is  the  simplest  modeling  within  lES/BTI.  Statistical  analysis  is 
used  when  there  are  known  discriminates,  exploitable  distributions,  and  non-complex 
constraints.  Statistical  analysis  is  used  extensively  in  Test  and  Evaluation  within  Ae 
lES/BTI  program  and  in  likelihood  ratio  (LR)  calculation.  Energy  minimization,  a  related 
teclmique  is  also  used  in  LR  calculation.  Generally  in  LR  calculaticms  the  hypo^eses 
features  are  compared  with  distributions  of  true  hypotheses  (numerator  of  LR)  and  false  or 
null  hypotheses  (denominator  of  LR).  Vehicle  detection  LR  is  an  example  of  the 
numerator/denominator  LR  calculation.  For  detections  using  current  distributions,  LRs  for 
military  vehicles  are  greater  to  or  equal  to  ten  (LR  >=  10)  for  ninety  percent  (90%)  of  the 
vehicles,  while  fifty  percent  (50%)  of  false  alarms  (non-militaiy  vehicle  detections)  have 
LR  of  one  or  less  (LR  <=  1).  Statistical  analysis  is  often  used  as  part  of  other  models  in 
lES/BTI. 

Models  of  forces  or  force  models  are  hierarchically  organized,  are  multidimensional, 
serve  as  the  basis  for  hypotheses  within  die  Bayes  netwOTk  and  have  characteristic  features. 
Force  models  have  two  explicidy  hierarchical  dimensions  force  level  (echelon)  and 


type/compositicMi  as  shown  in  Figure  3.  In  lES/BTI,  a  generic  type  unit  is  derived  to  serve 
as  parent  hypotheses  fw  specific  (artille^,  armor,  etc)  type  unit  hypotheses  and  act  as 
"untyped”  unit  hypotheses.  These  generic  units  have  characteristic^y  weaker  discriminate 
value  reflected  in  their  features.  Forces  have  in^licit  dimensions  such  as  military  situation 
and  deployment  which  are  treated  as  features.  A  list  of  potential  model  features  is  given  in 
Table  5. 


Figure  3.  Hierarchy  in  Force  Model  Type  and  Force  Echelon. 


Table  5.  Force  Model  Features 


Force  model  feature 

force  level 

type 

parent  unit(s) 

subunits  type/count 

total  vehicle  count 

deployment 

situation 

unit  width 

unit  length 

key(s) 

constraint(s) 

pattem(s) 

density 

uniformity  of  vehicle  LR 

curvature 

linearity 


Feature  type 
enumeration 
enumeration 
list 

distribution 

distribution 

distribution 

list 

distribution 

distribution 

list 

list 

list 

distribution 

distribution 

distribution 

distribution 


Constraints  and  evaluation  models  can  be  complex,  serve  several  functions  and  some 
are  hierarchical  in  nature.  Constraints  and  evaluation  nrodels  limit  the  hypotheses  space, 
provide  evidence  for  or  against  hypotheses  and  ultimately  influence  control  flow. 

Constraints  generally  set  the  context  in  which  a  hypothesis  will  be  entertained.  For 
example,  units  hypothesized  to  be  in  "road  halt"  have  j^esence  near  a  road  as  a  constraint. 

Evaluation  models  are  generally  the  functicm  that  maps  the  evidence  via  the  features  or 
attributes  of  a  hypothesized  force  to  a  evidence  value  (LR).  Evaluation  models  can  be 
simple  statistictd  analysis  or  a  complex  combination  of  constraints,  rules,  statistical 
analyses  and  energy  minimization  functions.  For  example,  evaluating  an  artillery  battery 
whose  pattern  is  warped  by  objective,  elevation,  treeline  and  field  of  fire  considerations 
would  take  a  complex  evaluation  model.  Some  of  the  statistical  distributions,  rules  and 
energy  minimization  functions  underlying  the  constraints  and  evaluation  models  are 
sensitive  to  adversary  and  theater  of  operation.  TherefOTe,  for  optimal  prerformance  these 
data  should  be  verified  for  each  adversary  and  theater  of  operation. 


Theoretic  control  models  are  used  to  control  the  computer  resources,  with  the  goal  of 
answering  the  image  exploitation  request  as  quickly  as  possible.  The  basis  of  lES/BTI 
control  decision  theory  is  pronwtion  of  the  highest  value  actions.  Value  is  a  function  of 
action  request  age,  the  action's  potential  payoff  and  the  action's  cost. 


Use  of  Multiple  Evidence  Sources 


Using  multiple  evidence  sources  provides  information  not  available  in  the  image, 
provides  stronger  evidence,  and  allows  more  complex  (and  realistic)  reasoning.  The  three 
distinct  evidence  sources  used  in  DES/BTI  are  imagery,  intelligence  information  and  terrain. 

Image  derived  information  spawns  hypotheses  and  generally  supplies  the  most 
evidence  for  a  hypotheses.  Image  derived  information  provides  most  of  the  metric 
information  about  a  unit,  includmg  vehicle  count,  inter-unit  and  intra-unit  spatial 
relationships. 

Intelligence  information  provides  situation  information  and  TacElint  data.  Situation 
information  can  provide  keys  for  force  models,  can  drive  searches  for  units  and  can 
influerxe  control  flow.  TacElint  can  provide  keys  fw  force  models,  can  drive  searches  for 
targets,  provide  evidence  for  forces  and  influence  control  flow. 

Terrain  provides  evidence  at  several  levels  such  as  eliminating  detections  in  no-go 
areas,  unit  hospitability  evaluation,  provide  constraints  and  evaluation  of 
forceMeployment/terrain  interactions. 

Interaction  between  the  different  evidence  sources  and  the  hypotheses  features  and 
attributes  can  be  quite  complex  and  the  constraint  and  evaluation  r^els  take  these 
interactions  into  account  Similarly  the  uniform  certainty  calculus  balances  the  relative 
value  of  evidence  from  each  evidence  node  within  the  Bayes  network.  Use  of  several 
evidence  sources  also  limits  the  number  of  (reasonable)  potential  hypotheses  by  using 
multi-source  constraints. 

Coarse-to-fine  Reasoning 

lES/BTI  uses  coarse-to-fine  hierarchical  reasoning  to  take  vehicle  detections  and  form 
them  into  the  highest  echelon,  most  refined  force  hypotheses  supported  by  the  image 
analysis.  Coarse-to-fine  reasoning  is  hierarchically  used  to  apply  constraints  at  the 
detection,  cluster  and  force  hypotheses  levels.  Evaluating  force  hypotheses  is  inherently  a 
coarse-to-fine  process  since  ffiS/BTI  competes  generic  and  typed  force  hypotheses. 
Coarse-to-fine  reasoning  avoids  rejection  of  true  units  by  allowing  generic  type  units  to  be 
gracefully  refined  into  typed  units  without  keeping  large  numbers  of  hallucinations  (false 
Sarms). 

Coarse-to-fine  reasoning  is  implemented  in  lES/BTI  at  several  levels.  Table  6 
illustrates  the  levels  of  constraints,  evidence  and  reasoning  within  lES/BTI.  The  levels  of 
reasoning  are  detection,  cluster,  force  models  and  complex  force  nnodels. 


Table  6.  Levels  of  Reasoning  within  lES/BTI 


Level _ Reasoning  Base _ 

I  Detection 

n  Ouster 

in  Force  model/force  hypothesis 

IV  Complex  force  modelTorce  hypothesis 

Within  the  lES/BTI's  processing  flow  shown  in  Figure  2,  the  coarsest  level  of 
constraints,  keys  and  evidence  are  applied  in  a  pipeline  fashion  (the  detect,  group,  military 
context/terrain  portion  of  Figure  2).  Table  7  identifies  the  constraints  and  keys  placed  on 
data  for  the  pipelined  actions  in  Figure  2.  LR  indicates  a  likelihood  ratio  is  computed  by 
the  action. 


Table  7.  Constraints  and  Keys  for  Pipelined  Actions 
LR  Constraints _ Keys 


Action 
Detection 
No-go  masking 
Clustering  X 


Hospitability  X 

TacEUni 


size 

detection  not  in  no-go  area 
min  number  of  detections 
max  number  of  detections 
detection  density 
detection  LR  >=  10 


Level 
I 

I 

n 
n 

II 
I 

near  roads  II 
TEhit  n 


As  shown  in  Figure  2,  hypotheses  are  formed  and  opportunistic  generation,  search, 
pruning  and  evaluation  occurs  after  the  pipelined  actions.  Table  8  shows  the  reasoning 
level  for  opportunistic  actions.  Group  subunits  into  larger  forces,  assess  force  type,  assess 
deployment  type  and  assess  formation  match  represent  the  next  level  of  coarse-to-fine 
reasoning.  Within  these  functions  force  models  arc  matched,  as  force  models  are  refined  in 
type,  situation  and  deployment  constraints  and  feature  distributions  arc  tightened. 

Table  8.  Levels  of  Reasoning  for  Opportunistic  Actions 


Action 

Group  Subunits 
Assess  force 
Assess  formation 
Assess  deployment 
Force  pi^ction 
Force  SRA 


Level _ Reasoning  Base _ 

HI  Force  model/force  hj^thesis 

ni  Force  model/force  hypothesis 

in  Fwce  model/force  hypothesis 

in  Force  model/force  hypothesis 

IV  Complex  force  modeVforce  hypothesis 

IV  Complex  force  model/force  hypothesis 


Finally,  spatial  region  adjustment  (SRA)  and  force  location  prediction  represent  the 
highest  levels  of  coarse-to-fine  reasoning.  Level  IV  actions  use  complex  force 
models/force  hypotheses.  These  complex  models  often  use  more  than  one  evidence  source 
(imageiy,  terrain,  TacElint)  and  often  use  energy  minimization  functions  to  balance 
conflicting  influences.  Level  FV  actions  are  relatively  more  costly  to  perform  and  their  use 
is  limited  by  the  constraints  required  by  the  models. 

Cpngpl  FlQffi 

The  fourth  cornerstone  of  lES/BTl  processing  philosophy  is  a  flexible  control  flow.  A 
flexible  control  flow  allows  efficient  exploitatitm  of  the  task  and  the  ability  to  efficiently 
process  for  each  theater  and  adversary.  TTie  flexible  control  flow  is  supported  by  Doers 
and  Directed  Acyclic  Graphs  (DAG).  Doers  and  DAGs  essentially  perform  "action 
planning"  and  some  are  dynamic.  For  flexible  control  DAGs  can  be  predefined,  stored  in 
libraries  and  called  when  needed.  In  this  way,  the  system  can  be  ran  by  several  different 
users  each  with  a  different  control  flow. 

Opportunistic  control  is  also  part  of  the  flexible  control  flow.  As  shown  in  Figure  2, 
the  first  portion  of  EES/BTI  processing  is  pipelined.  However,  once  hypotheses  are 
formed  opportunistic  action  planning  starts.  The  concept  is  to  allow  the  state  of  the 
hypotheses  within  the  Bayes  network  to  drive  further  processing  by  suggesting  actions 
with  an  associated  potential  payoff  to  the  Decision  Theoretic  Controller  ^TQ  for 
execution.  The  DTC  uses  the  age,  potential  payoff  and  "cost"  to  schedule  tasks.  The 
process  is  dynamic  as  tasks  return  evidence  to  the  Bayes  network  and  the  Bayes  Net 
Manager  (BNM)  updates  the  Bayes  network’s  state  and  then  suggests  more  actions  to  the 
DTC.  Processing  stops  when  DTC  determines  sufficient  evidence  has  been  gathered  to 
answer  the  exploitation  request  and  the  infcHmation  is  forwarded  to  the  analyst. 

Bayesian  Inference 

Bayesian  inference  is  the  method  lES/BTI  uses  to  perform  higher  level  inference. 
Bayesian  inference  produces  a  single  belief  for  each  hypothesis,  correlates  evidence  with 
hypotheses  within  a  Bayes  network  node,  correlates  evidence  between  Bayes  networit 
nodes  and  allows  estimation  of  (potential)  payoff  for  actions.  The  BNM  uses  Bayesian 
inference  as  an  integral  part  of  building  and  updating  the  Bayes  network  and  for  suggesting 
actions  to  the  DTC.  Pearl's  algorithm  is  the  basis  of  in^lementation  within  lES/BTI. 

Bayesian  inference  is  based  upon  the  theory  that  given  a  set  of  hypotheses  H,  that  are 
mutually  exclusive  and  exhaustive,  subsets  of  H  can  have  semantic  interest  and  form  a 
hierarchy  related  through  a  parent.  The  effect  of  evidence  on  the  hypotheses  tree  or  Bayes 
network  can  then  be  defined  by  a  formalism  involving  estimation,  weight  distribution  and 
belief  updating.  Thus  Bayesian  inference  allows  evidence  to  be  introduced  at  any  node  in 
the  Bayes  network  and  the  evidence's  effect  can  be  appropriately  distributed  within  the 
Bayes  network. 


TEST  &  EVALUATION 


System  Test  &  Evaluation  (T&E)  is  being  perfonned  on  lES/BTI  version  1.5.  The 
testing  of  version  1 .5  is  scheduled  to  be  performed  in  three  phases  as  shown  in  Table  9. 

Table  9.  Phases  of  lES/BTI  Test  &  Evaluation 


Phase 

Description 

Imag 

1 

Tuning 

18 

1 

System  Blind  Test 

33 

2 

Tuning 

51 

2 

System  Blind  Test 

>60 

3 

Component  Level  Analysis 

>60 

3 

Tuning 

>111 

3 

Component  Level  Analysis 

>111 

Phase  1  includes  a  tuning  period  where  developers  are  allowed  to  adjust  system 
parameters  based  on  experimentation  using  18  images.  Phase  1  also  includes  a  blind  test 
on  33  images.  None  of  the  blind  test  images  spatially  overlap  the  tuning  images. 

Phase  2  includes  a  tuning  period  where  developers  are  allowed  to  adjust  system 
parameters  based  on  experimentation  using  the  full  set  of  Phase  1  imagery  (51  images). 
Phase  2  also  includes  a  blind  test  on  over  ^  images.  None  of  the  Phase  2  blind  test 
images  spatially  overlap  the  Phase  1  images. 

Phase  3  performs  component  level  evaluation.  In  Phase  3  results  from  Phase  2  are 
evaluated  at  the  component  level.  Then  a  system  tuning  is  perfonned  based  on  the  entire 
set  of  imagery  and  component  analysis  is  repeated.  Phases  3  serves  as  a  base  on  which 
enhancements  to  version  1.5  can  be  evaluate 

Image  Truth 

Image  truth  serves  as  the  basis  of  comparison  for  the  T&E  effort.  Two  types  of  image 
truth  were  created  "SAR  only"  and  "all  source."  SAR  only  is  an  LA's  interpretation  based 
solely  on  the  radar  image.  All  source  image  truth  is  based  on  all  the  available  information 
with  one  exception,  vehicles  were  only  detected  based  on  the  SAR  image.  This  was  done 
to  avoid  hallucinating  vehicles  which  may  have  moved.  However,  all  source  information 
could  type  vehicles  (APC,  tank,  etc)  detected  in  the  SAR  image. 

In  creating  the  image  truth,  the  analysts  were  not  constrained  to  assigning  vehicles  to 
companies,  companies  to  battalions,  etc.  Analysts  could  identify  elements  of  units, 
unassigned  vehicles,  single  companies  or  whatever  they  believed  the  correct  interpretation 
of  the  data  to  be. 


System  Response 


System  response  is  lES/BTI's  estimate  of  the  forces  present  in  the  image.  System 
response  atten^ts  to  minimize  omission  eiiors  (misses)  and  commission  errors  (false  alarm 
units)  and  tries  to  graphically  present  units  in  the  image  to  the  lA.  System  response  tries  to 
present  an  uncluttered  grapluc  and  tends  to  be  conservative  when  displaying  unit  type. 
Figtire  4  is  an  example  of  the  graphic  presentation  to  the  analyst  System  response  is  the 
lES/BTI's  produced  basis  for  system  T&E. 


Figure  4.  Example  of  System  Response  Graphic  Overlay. 

In  this  example,  2  untyped  battalions,  a  untyped  company  and  a  artillery  battery 
are  presented  to  the  an^yst. 


Tests 

Several  tests  were  devised  to  automaticaUy  score  the  system.  The  "50  percent  test" 
tests  whether  system  response  sufficiendy  presents  units  for  an  analyst  to  identify  it 
Under  the  50  percent  test  a  "hit"  was  defin^  as  greater  or  equal  to  fifty  percent  (>=50%)  of 
an  image  truth  unit's  vehicles  were  identified  with  EES/BTTs  system  response. 

The  "null  test"  tests  whether  system  response  totally  fails  to  identify  a  unit.  The  null 
test  defined  a  "miss"  as  zero  (0)  of  the  image  truth's  vehicles  are  identified  with  lES/BTI's 
system  response.  A  "miss"  does  not  imply  die  vehicles  were  not  detected,  only  that  any 
hypotheses  containing  them  were  not  part  of  the  system  response. 

The  "hallucination  test"  tests  whether  system  response  hypotheses  are  entirely 
composed  of  false  vehicle  detections  and  therefOTe  is  a  system  generated  hallucinaticm. 
Hallucinations  are  defined  as  hypotheses  presented  as  part  of  the  system  results  containing 


no  image  truth  vehicles.  Hypotheses  containing  "unassigned"  image  truth  vehicles  or 
elements  of  units  are  not  considered  hallucinations. 

Results 

Phase  1  of  the  T&E  effort  is  complete  aiKi  lES/BTI’s  system  response  performance 
against  all  source  image  truth  are  given  in  Table  1 1  and  Table  12.  Table  1 1,  shows  how 
well  lES/BTI's  system  response  identified  all  source  units.  lES/BTI  is  designed  to  find 
battalion  and  hi^er  level  forces  but  also  identifies  companies.  Under  the  50  percent  test, 
lES/BTI  identified  both  brigades  by  identifying  their  sub-forces.  For  battalions  lES/BTI 
performed  significantly  worse  missing  27  percent  of  the  image  truth  units  and  identifying 
only  61  percent  in  Phase  1  testing.  12  percent  of  the  battalions  had  some  subunits 
identified  but  failed  the  50  percent  test 

Table  1 1.  Phase  1  Hits  &  Misses 


Unit  level 

rr 

Hits 

Partial  hits 

Misses 

Elements 

20 

- 

Companies 

63 

36  (57%) 

3(5%) 

24  (38%) 

Battalions 

74 

45  (61%) 

9  (12%) 

20  (27%) 

Brigades 

2 

2  (100%) 

0 

0 

Table  12  shows  how  many  lES/BTI  system  response  hypotheses  were  hallucinations. 
lES/BTI  system  response  did  not  produce  any  brigade  level  hypotheses  and  therefore 
produced  no  brigade  hallucinations.  13  percent  of  the  battalion  level  hypotheses  were 
found  to  be  hallucinations.  Note  "lES/BTI  total"  of  Table  12  includes  partially  correct 
hypotheses  such  as  a  company  hypotheses  composed  of  image  trath  "unassigned"  vehicles. 
Also  note  in  Table  1 1  and  Table  12  subunits  of  a  higher  force  are  not  includ^  in  unit  level 

Table  12.  Phase  1  Commission  Errors 

Unit  level  lES/BTI  total  Hallucinations 

Ccrapanies  155  31  (22subunit) 

Battalions  87  11(13%) 

Brigades  0  0 


Analysis 

Testing  is  not  far  enough  along  to  give  insightful  analysis  and  totals  including  subunits 
is  not  complete.  However,  some  observations  can  be  made.  The  image  data  set  included  a 
few  null  images  (containing  no  units)  and  system  response  was  restrained  in  hallucinating 
units.  Miss^  units  and  hdlucinations  were  distributed  throughout  the  imagery  set 
indicating  a  even  response  across  the  imagery.  At  the  battalion  level,  misses  were  spread 
across  unit  type,  but  at  the  (independent)  company  level  the  majority  of  missed  companies 


were  support  This  may  indicate  that  by  adjusting  the  threshold  at  which  hypotheses  are 
present^  as  part  of  system  results,  performance  at  the  battalion  level  may  be  improved. 
Tuning  with  a  larger  data  set  for  Phase  2  testing  and  using  information  from  Phase  1  testing 
should  improve  system  response’s  performance  in  Phase  2. 
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